Beyond pairwise strategy updating in the prisoner's dilemma game 
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In spatial games players typically alter their strategy by imitating the most successful or one randomly se- 
lected neighbor. Since a single neighbor is taken as reference, the information stemming from other neighbors 
is neglected, which begets the consideration of alternative, possibly more realistic approaches. Here we show 
that strategy changes inspired not only by the performance of individual neighbors but rather by entire neigh- 
borhoods introduce a qualitatively different evolutionary dynamics that is able to support the stable existence 
of very small cooperative clusters. This leads to phase diagrams that differ significantly from those obtained 
by means of pairwise strategy updating. In particular, the survivability of cooperators is possible even by high 
temptations to defect and over a much wider uncertainty range. We support the simulation results by means of 
pair approximations and analysis of spatial patterns, which jointly highlight the importance of local information 
for the resolution of social dilemmas. 



Cooperative behavior is extremely important, both in the 
animal world as well as across human societies 111-0]. Yet it is 
also an evolutionary puzzle, as it is costly but has no immedi- 
ate individual benefits, except in rare exceptions, for example 
when cooperation is agreed upon as a risk-sharing mechanism. 
How cooperation evolved amongst selfish and unrelated indi- 
viduals is therefore still ardently investigated, as evidenced by 
recent reviews [iShIOiI. 



Evolutionary game theory 111 11 - 11311 provides an apt theoret- 
ical framework to address the subtleties of the evolution of 
cooperation. One of the most popular games that is represen- 
tative for situations constituting a social dilemma is the pris- 
oner's dilemma game 111]. It can be summarized succinctly. 
Two individuals have to decide simultaneously whether they 
wish to cooperate or not. Cooperator pays a cost c towards the 
mutual benefit b where 6 > c > 0, while defector contributes 
nothing. This yields the temptation to defect T ^ b, reward 
for mutual cooperation R = b — c, punishment for mutual de- 
fection P = 0, and the sucker's payoff 5 = — c, which for the 
prisoner's dilemma game thus satisfy T > R > P > S and 
2R > T + S. Evidently, for an individual it is best to defect 
regardless of what the opponent does. As rational players are 
aware of this, they both defect, in turn obtaining P rather than 
R, hence the social dilemma lITill . 

Several mechanisms that facilitate the evolution of cooper- 
ation are known. Nowak summarizes five rules i^, which are 
kin selection ifTsll . direct reciprocity lfl6ll . indirect reciprocity 
iHtIi . group selection ifisll . and network reciprocity lfl9[l . Net- 
works in particular, have received substantial attention in the 
recent past f^. While scale-free networks appear to provide 
the best environment for the evolution of cooperation 1201427 
small-world ll28l432ll and hierarchical networks ll33l-[35ll also 
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received ample attention. Largely motivated by the discovery 
that complex networks facilitate the evolution of cooperation, 
heterogeneity in general has emerged as an important property 
that may help keep defectors in the minority Il36l - B9ll . Coevo- 
lutionary games tlOi1 . where the structure of the network was 
subject to evolution just as the strategies of players have been 
studied thoroughly too ll40l - l54ll . with the prevailing conclu- 
sion being that this may give rise to robust cooperative states 
and lead to socially preferable interaction networks in a spon- 
taneous manner Quite remarkably, this has recently been con- 
firmed empirically ifssll . although very extensive experiments 
also indicate that the human behavior may suppress network 
reciprocity ll56il57ll . 

In fact, how human decision-making affects the evolution 
of cooperation is of particular relevance for the present work. 
Szabo et al. ifssll have recently considered a special type of 
strategy updating. Instead of players exclusively caring only 
about their own payoffs when updating their strategies, they 
investigated what happens when a pair of randomly chosen 
neighboring players tries to maximize their collective income 
by simultaneously updating their two strategies. It was re- 
ported that the proposed strategy update rule produces the an- 
tiferromagnetic ordering structure of cooperators and defec- 
tors on the square lattice at sufficiently low noise intensities, 
and that this favors the evolution of cooperation more than 
the traditional pairwise imitation updating. Human decision- 
making dynamics has also been investigated experimentally, 
whereby we are particularly interested in the so called "social 
influence" effect reported by Lorenz et al. lf59ll . As stated in 
their paper, social influence among group members plays an 
important role in individual decision-making. 

One may then ask how this affects the evolution of coop- 
eration? To address this question, we propose an adaptive 
strategy-adoption rule in which the social influence is taken 
into account. In particular, as a proxy for the social influence 
we assume that the decisions the players make are affected 
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FIG. 1 : Fraction of cooperators pc as a function of the cost-to-benefit 
ratio r, as obtained for K = G.l [panels (a) and (b)] and K = 0.83 
[panels (c) and (d)]. Results presented in panels (a) and (c) were ob- 
tained by means of Monte Carlo simulations, while those presented 
in panels (b) and (d) were obtained by means of pair approximation 
(see Methods section for details). Figure legend indicates whether 
pairwise or locally influenced strategy updating was used. 



by all their neighbors, not just a single randomly selected or 
the most successful neighbor. Players can collect information 
from their neighbors, and moreover, their decision-making 
is more likely to be affected by the circle of "close friends" 
rather than the whole social environment. We introduce this 
so-called local influence to the strategy updating simply so 
that, before a potential update, each player considers the av- 
erage performance of its own strategy and that of the other 
strategy, if present, within its neighborhood. As we will show 
in what follows, this introduces a qualitatively different evo- 
lutionary dynamics that is able to support the stable existence 
of very small cooperative clusters, which in turn supports the 
survivability of cooperative behavior even under very unfa- 
vorable conditions. Besides simulation results ll60ll , we will 
also present results obtained with pair approximation meth- 
ods, which are, along with the game theoretical model, accu- 
rately described in the Methods section. 



Results 

We begin by presenting the fraction of cooperators pc as a 
function of the cost-to-benefit ratio r = c/h at two tempera- 
tures, namely at K = 0.1 and K ~ 0.83. The usage of the 
latter value is motivated by recent empirical research from be- 
havioral science ll6lll . although essentially, as we will show 
in what follows, the temperature, i.e., the level of uncertainty 
by strategy adoptions, does not play a decisive role. Results 
for both the pairwise and locally influenced strategy updat- 
ing are presented in Fig. [Tta^c). It can be observed that for 
-ft' = 0.1 the evolution of cooperation is promoted across the 
whole applicable span of r if the traditionally used pairwise 
strategy updating is replaced by the proposed local influence 
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FIG. 2: Full K — r phase diagrams, as obtained by means of Monte 
Carlo simulations [panels (a) and (c)] and by means of pair approx- 
imation [panels (b) and (d)]. Upper red (lower blue) lines denote 
the boundaries between the mixed C + D and homogeneous D (C) 
phases. 



based strategy updating. For K ~ 0.83, however, the outcome 
is a bit less clear-cut. While pairwise imitation fails to sus- 
tain cooperative behavior at such high values of r as locally 
influenced strategy updating, it is nevertheless more apt for 
achieving complete cooperator dominance. As we will show 
in what follows, it is indeed the case that locally influenced 
strategy updating often fails to completely eliminate defectors 
at small values of r, yet it opens up the possibility of survival 
of cooperators even under harsh defector- friendly conditions. 

These simulation results can be corroborated by results 
of pair approximations (see Methods for details), which we 
present in Fig. [nb,d). The general trends are predicted cor- 
rectly, although as expect, the beneficial effect of network 
reciprocity (19] at low values of r are underestimated. It is 
worth mentioning that the pair approximation is in general 
more accurate for larger values of K ll62ll . and indeed it can 
be observed that the agreement with simulation results is bet- 
ter for K ~ 0.83 than it is for K = 0.1. In particular, for 
K ~ 0.83 the pair approximation method correctly predicts 
the occurrence of an intersection point [compare panels (c) 
and (d)]. Altogether, results of pair approximations corrobo- 
rate the conclusion that the survivability of cooperators, espe- 
cially at high values of r, is substantially promoted by locally 
influenced strategy updating. 

Further adding to the robustness of this conclusion are re- 
sults presented in Fig. |2a,c), where we present full K — r 
phase diagrams for both considered updating rules. It can be 
observed that the positive impact of local influence on the evo- 
lution of cooperation persists across large regions of K. On 
the other hand, the presented phase diagrams also evidence 
more clearly the failure of the proposed updating rule to lead 
to an absorbing C phase. Moreover, there is a notable quali- 
tative difference in the critical behavior that is evoked by the 
updating rule. By focusing on the -D — > C + -D phase bound- 
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aries, it can be observed that for pairwise strategy updating 
there exists an optimal value of K at which cooperators thrive 
best. Note that the D — ;> C+D phase boundary is bell-shaped, 
indicating that K w 0.3 is the optimal temperature at which 
cooperators are able to survive at the highest value of r. For 
strategy updating based on local influence, however, this fea- 
ture is absent. The D ^ C + D phase boundary is in fact an 
inverted bell, indicating the existence of the worst rather than 
an optimal value of K. Notably, the results for pairwise strat- 
egy updating are in agreement with previous works ll62|j64ll . 
where it was shown that the lack of overlapping triangles, as 
is the case for the square lattice as well as for random regu- 
lar graphs, introduces an optimal uncertainty for the evolution 
of cooperation. Conversely, the results obtained by consid- 
ering local influence suggest that the system is behaving as 
if overlapping triangles were in fact present in the interaction 
network. Note that in the latter case an optimal K for the 
evolution of cooperation does not exist. This leads us to the 
conclusion that the interaction network is effectively altered 
when the local influence is taken into account. In particu- 
lar, triplets of players that are not connected by means of the 
original interaction graph (the square lattice) become effec- 
tively connected through the joint participation of players in 
the same local groups (neighborhoods) that are subject to the 
same local influence. An identical effect was indeed observed 
by the study of the public goods game ll65ll . where triplets 
also became effectively connected because of the participa- 
tion of players in the same groups. Below, we will provide 
further evidence concerning the effective linkage of triples of 
players, which is essentially a side effect of locally influenced 
strategy updating. Another interesting observation is that the 
parameter region of the mixed C + D phase in general widens 
as K increases, which is in contract to the results obtained by 
means of pairwise strategy updating. 

We have also constructed full K — r phase diagrams by 
means of pair approximations. Figure[2jb,d) features the ob- 
tained results, from which it follows that qualitative features, 
compared to the simulation results, are again captured fairly 
accurately, although the extent of the parameter region of 
the mixed C + D phase is overestimated. Expectedly, the 
predictions are also less accurate near the phase boundaries, 
which is because the pair approximation does not take into 
account loops nor does it take into account long-range corre- 
lations, which however, have a noticeable effect especially in 
the vicinity of critical transitions ll66ll . 

In order to obtain an understanding of the reported observa- 
tions, we proceed with the presentation of characteristic spa- 
tial patterns, as obtained for both pairwise and locally influ- 
enced strategy updating, in Fig. [3] Regardless of which update 
rule is applied, cooperators form compact clusters by means of 
which they are able to exploit the mechanism of network reci- 
procity 111911 . If the value of r is small, the clusters are larger 
and more compact than for higher values of r. On the other 
hand, the spatial patterns emerging under the two update rules 
also have noticeable dissimilarities. Foremost, given a value 
of r, pairwise strategy updating yields larger clusters than lo- 
cally influenced strategy updating, even if the density of co- 
operators is approximately the same [compare panels (a) and 




FIG. 3: Characteristic snapshots of spatial patterns formed by co- 
operators (blue) and defectors (red) under pairwise imitation [(a) 
r = 0.004, (b) r — 0.019] and under strategy updating based on lo- 
cal influence [(c) r = 0.004, (d) r = 0.221]. The size of the square 
lattice was 100 x 100 and K = 0.83. (a) In this snapshot there 
are 77 clusters, ranging in size from a single cooperator to 3042 co- 
operators, with a weighted average size of 1925.21. The stationary 
fraction of cooperators is pc ~ 0.52. (b) In this snapshot there are 
99 clusters, ranging in size from a single cooperator to 162 coopera- 
tors, with a weighted average size of 70.01. The stationary fraction 
of cooperators is pc ~ 0.19. These characteristics are significantly 
different in the bottom two snapshots, (c) In this snapshot there are 
439 clusters, ranging in size from a single cooperator to 427 coopera- 
tors, with a weighted average size of 137.69. The stationary fraction 
of cooperators is pc ~ 0.52. (d) In this snapshot there are 164 clus- 
ters, ranging in size from a single cooperator to 19 cooperators, with 
a weighted average size of 6.63. The stationary fraction of cooper- 
ators is pc ~ 0.05. Note that in snapshots (a) and (c) the densities 
of cooperators for both update rules are practically identical, while 
nearer to the extinction thresholds [panels (b) and (d)] they differ 
quite significantly. 



(c)]. Nearer to the extinction threshold the stationary densities 
differ, yet the difference in the spatial patterns the two rules 
generate becomes most apparent [compare panels (b) and (d)]. 

The visual inspection of the characteristic spatial patterns 
invites a quantitative analysis of the exposed differences, the 
results of which are presented in Fig. |4] separately for both 
updating rules. It can be observed that, in general, as r in- 
creases, the cluster size decreases. The number of clusters, on 
the other hand, is maximal at an intermediate value of r. Con- 
crete r values, however, differ significantly for the two consid- 
ered strategy updating rules. In particular, by pairwise strat- 
egy updating both the clusters size and the number of clusters 
are shifted significantly towards lower values of r. One rea- 
son is obviously that pairwise strategy updating simply does 
not support the survivability of cooperators by as high val- 
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FIG. 4; Macroscopic properties of cooperative clusters in the depen- 
dence on the cost-to-benefit ratio r. Cluster size (a) and cluster count 
(b) are depicted for pairwise and locally influenced strategy updat- 
ing. In both cases the cluster size decreases as r increases, while the 
cluster count reaches a maximum at a certain value of r and then de- 
creases. Note that for pairwise imitation a minimum cluster size of 
about 76.18 is required for cooperators to survive. Taking into ac- 
count the local influence of the neighbors reduces this to 6.61. The 
depicted results were determined in the stationary state on 100 x 100 
square lattices and by using K = 0.83. Error bars indicate the stan- 
dard deviation. 



ues of r as locally influenced strategy updating. Nonetheless, 
the fact that for any given value of r, where comparisons are 
possible, the typical cluster size obtained with pairwise strat- 
egy updating is much larger than the one obtained with lo- 
cally influenced strategy updating begets the conclusion that 
there are significant differences in the way cooperators clus- 
ter to withstand being wiped out by defectors. Note that for 
cooperators to survive under pairwise updating the minimally 
required cluster size is « 76.18, while for locally influenced 
updating it is only 6.61. Moreover, for pairwise strategy up- 
dating the cluster size decreases much faster, which speaks in 
favor of the increased stability of the clusters under locally 
influenced strategy updating. 

To confirm these conjectures, we present in Fig.|5]two typ- 
ical C-cluster configurations and analyze the survivability of 
cooperators separately for each particular case. For the sake of 
simplicity but without loss of generality, we consider for the 
following analysis only the A' — > limit. Then if the payoff of 
each cooperator along the boundary is larger than that of each 
defector in its neighborhood, we are allowed to conclude that 
such a C-cluster will survive. For the left C-cluster pattern in 
Fig.|5]under pairwise updating, the payoffs of a cooperator C 



(Pc) and defector D (Pd) along the boundary are 

Pc = 2 and Pz5 = l + 4r, (1) 

respectively. For locally influenced updating, however, the 
average payoff of cooperators (Pc) and the average payoff of 
defectors (Pd) along the boundary are given by 



Pc = 2 and Pd = 1 + 4r, 



(2) 



respectively. Thus for such a C-cluster pattern to survive, both 
update rules lead to r < —0.25. Indeed, neither locally influ- 
enced nor pairwise strategy updating support the survivability 
of such a pattern. Performing the same analysis for the config- 
uration on the right, however, yields a different outcome. The 
payoff of a cooperator C2 (Pc2 ) on the boundary and that of 
the two types of defectors Di and D2 (Pd2 and Pd^ ) are 



Pc, = 1, Pdi = 2 + 4r and Pd^ = 1 + 4r, 



(3) 



respectively. For locally influenced updating the correspond- 
ing payoffs are 



^andPD^- 



4r. 



(4) 



Accordingly, we find that under pairwise updating the con- 
dition for survivability is r < —0.25, while under locally 
influenced updating it is only r < Hence, locally influ- 
enced strategy updating can warrant the survivability of co- 
operators when grouped in this way, while pairwise updating 
can not. Note also that the C-cluster configuration on the right 
of Fig. |5]is the smallest one which can persist in the popula- 
tion under the most hostile conditions under locally influenced 
strategy updating. Based on this analysis, we can in fact es- 
timate the extinction threshold r ^ 0.21 in the limit 
K —5- 0, and indeed we find excellent agreement between this 
analytical approximation and the simulation results presented 
in Fig. He). 

With these insights, we argue that local influence based 
strategy updating can support the survivability of cooperative 
behavior only if the core of the C-cluster is isolated from de- 
fectors (compare left and right configuration of Fig. |5]l, be- 
cause cooperators along the boundary can then gain a higher 
level of support from the cluster and thus protect themselves 
against being exploited by defectors. In previous works, 
where only pairwise strategy updating was considered, indi- 
vidual players were concerned only with their own payoffs 
when updating their strategies. However, if individuals are ex- 
posed to the local influence, i.e., they care about the average 
performance of the strategies in their neighborhood, coopera- 
tors can benefit not only from their own payoffs, but also from 
the payoffs of their cooperative neighbors. In this sense, lo- 
cally influenced strategy updating further strengthens the link- 
age between cooperators within cooperative clusters, and so 
cooperators can reciprocate with each other on a profounder 
and altogether more effective level. 

In terms of the robustness of the described mechanism to 
variations of the interaction network, our preliminary investi- 
gations indicate that cooperation is always promoted on regu- 
lar small-world networks with different rewiring probabilities 
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FIG. 5: Schematic presentation of two representative cooperative 
(blue) clusters surrounded by defectors (red). The cluster depicted 
left has no chances of survival under pairwise or locally influenced 
strategy updating. The cluster on the right, however, cannot prevail 
under pairwise imitation, but can do so under locally influenced strat- 
egy updating. This is because the core of the cooperative cluster (Ci 
in the figure) is quarantined from defectors in case imitation proceeds 
according to local influence (see main text for details). 

ll67ll . as well as on scale-free networks ll68ll provided the pay- 
offs are normalized with the number of neighbors jzsll . If the 
payoffs on scale-free networks are not normalized with the 
number of neighbors ll20ll . the promotion of cooperation due 
to local influence, compared to the traditional pairwise strat- 
egy updating, may be compromised. Additional research is 
needed, however, to clarify conclusively the potential nega- 
tive impact of strongly heterogeneous degree distributions on 
the newly identified mechanism for the promotion of coop- 
eration. It would also be of much interest to clarify the role 
of zero-determinant strategies ll69l[70ll . which point to major 
paradigm shifts in the resolution of social dilemmas. 



Discussion 

Summarizing, we have analyzed the impact of "local influ- 
ence" on the evolution of cooperation in the spatial prisoner's 
dilemma game. Instead of the performance of a single neigh- 
bor, players considered the average performance of the two 
strategies within their neighborhoods. We have shown that 
by going beyond the traditionally assumed pairwise strategy 
updating, the evolution of cooperation can be promoted. We 
have determined full K — r phase diagrams by means of sim- 
ulations and pair approximation methods, which both indicate 
that this effect is robust against uncertainty by strategy adop- 
tions. Moreover, the phase separation lines indicate that the 
consideration of local influence effectively changes the inter- 
action network as an optimal K is no longer inferable. This is 
characteristic for interaction networks with overlapping trian- 
gles ll62ll64ll . which are obviously not part of the square lattice 
topology that we have employed. By analyzing the macro- 
scopic features of emerging spatial patterns as well as the sur- 
vivability of typical cooperative clusters, we have provided 
further insights as to how the consideration of local influence 
changes the evolutionary dynamics. 

Lastly, it is worth relating the presently considered strategy 



updating rule to previous game-theoretical models. By the 
win-stay-lose-shift rule l32l I7ll - l74ll . for example, each indi- 
vidual has an aspiration according to which it judges whether 
or not to change strategy. The aspiration, however, is tradi- 
tionally assumed to be constant. In our case, on the other 
hand, we relax this assumption by considering the aspiration 
as a dynamical quantity. Note that the average payoff of the 
strategy that is not adopted by the focal player can in fact be 
regarded as the aspiration level. This in turn implies that here 
the aspiration depends on the outcome of the game, and hence 
is subject to change. Moreover, the present rule can be re- 
garded as a learning rule. The difference from the traditional 
single role model learning rule is that in the present case the 
strategy update depends not on the comparison of a pair of in- 
dividuals, but on the comparison of two groups of individuals, 
each involving several individuals adopting the same strategy. 
Overall, we hope that these considerations, and in particu- 
lar the consideration of local influence and the "wisdom of 
crowds" I59, 75, 76], will motivate further research aimed at 
promoting our understanding of the evolution of cooperation. 

Methods 

Mathematical model 

Players are located on the vertices of a L x L square lattice 
with periodic boundary conditions. Each individual is initially 
designated either as a cooperator C or defector D with equal 
prob ability. For the pairwise imitation strategy updating rule 
IITtIi (we use the label "pairwise" in the figure legends when 
applying this rule), Monte Carlo simulations of the game are 
carried out comprising the following elementary steps. First, a 
randomly selected player x collects its payoff by interact- 
ing with its four nearest neighbors. For the purpose of payoff 
evaluation, it is worth introducing unit vectors S — [1,0]"^ 
and [0, 1]^ for cooperators and defectors, respectively. The 
payoff matrix is 



where r G (0, 1) is the cost-to-benefit ratio. The payoff of 
player x is thus 

Px= S^MS,, (5) 

z6r(x) 

where r(a;) represents its neighborhood. Subsequently, one 
randomly chosen neighbor y of player x also acquires its pay- 
off Py identically as previously player x. 

After the evaluation of payoffs, players consider changing 
their strategies. In particular, player x adopts the strategy Sy 
of the randomly selected neighbor with the probability 

where K is the uncertainty by strategy adoptions f^. If the 
local influence is taken into account (we use the label "local" 
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in the figure legends when applying this rule), however, the 
elementary steps are as follows. First, we randomly choose 
a player x with the strategy Sx- Next, we evaluate the aver- 
age payoff Ps-^ of those players who adopt the same strategy 
Sx, as well as the average payoff Pg^ of those players who 
adopt the opposite strategy Sx of player x, if any, within the 
neighborhood. In particular, we have 



and 



E PJiS^S,,) + Px 

E s{s^s,) + i 



E P.sis^Sz) 
E sis^Sz) ' 



(7) 



(8) 



where the Dirac delta function 6{x) satisfies 



6ix) 



0, iix ^ 

1, if a; = 



Lastly, player x will adopt the strategy Sx with the probability 

1 



l + eM'iPs.-PsJ/KY 



(9) 



where K is, as by pairwise imitation, the uncertainty by strat- 
egy adoptions. 

The presented simulation results were obtained by using 
L = 100 — 400 depending on the proximity to phase separa- 
tion lines and the size of the emerging spatial patterns. In ac- 
cordance with the random sequential update, each full Monte 
Carlo step, which consists of repeating the elementary steps 
L X L times corresponding to all players, gives a chance once 
on average for every player to alter its strategy. The stationary 
frequency of cooperators pc is determined by averaging over 
10"* Monte Carlo steps in the stationary state after sufficiently 
long relaxation times. In general, the stationary state has been 
considered to be reached when the average of the cooperation 
level becomes time-independent. To further increase the ac- 
curacy of our simulations, we have averaged the final outcome 
over 50 independent initial conditions. 



Pair approximations 

Let PC and pu = 1 — pc denote the frequencies of co- 
operators and defectors, respectively, and let pcc, Pen, Pdc 
aadpno represent the frequencies of CC, CD, DC and DD 
pairs, respectively. Then qx\Y = Pxy/py with X,Y e C, D 
specifies the conditional probability to find an X-player given 
that the neighboring node is occupied by an F-player. Note 
that here X, Y and Z denote either C or D. Instead of the 
first-order approximation considering the frequency of strate- 
gies as in the well-mixed population, the pair approximation 
tracks the frequencies of strategy pairs pxv {X^ F G C, D). 



The probabilities of larger configurations are approximated 
by the frequencies of configurations not more complex than 
pairs. Based on the compatibility condition px = 'YlyPxY, 
the symmetry condition p^y = Pyx, and closure conditions, 
Pc and Pec can fully determine the dynamics of the system. 
While the pair approximation for pairwise imitation is well- 
known and can be looked up for example in the Appendix of 
^ or more recently iItsIi . for the imitation based on local in- 
fluence the derivations are as follows. 

A defector is selected for strategy updating with the prob- 
ability p£i. Let kc and kn denote the number of cooperators 
and defectors amongst the neighbors on a regular lattice with 
degree k, respectively. The frequency of such a configuration 
is 



fe! kc ko 



(10) 



and the payoff of the defector is Pd {kc, ko) = (1 + r) • kc + 
r-ku. The configuration probability with which a neighboring 
cooperator has k(j cooperators and fc^ defectors as its neigh- 
bors is 



k'c\k'j,\ '^C\CDiD\CD^ 



(11) 



where qx\YZ gives the conditional probability that a player 
next to the Y Z pair is in state X. The payoff of the neighbor- 
ing cooperator is Pc{kfj, kjj) = k^. Similarly, the configu- 
ration probability with which a neighboring defector has k^ 
cooperators and kjj defectors as its neighbors is 



(fc-1)! 4 4 
k'c\k'j,\ '^c\dd1d\dd^ 



(12) 



and the payoff of the neighboring defector is P^^k^j, kj^) = 
{1 + r) ■ k^, + r ■ {kjj + 1). Thus, the average payoff Pc of 
cooperators that are neighbors of the focal defector is 



fc-i 



Pr. 



^ k' i.k'' i.'^C\CDlD\CD'-Pc{kc,kjj) 



(13) 



(fc - 1) • qc 



CD- 



The average payoff Pd of defectors that are neighbors of the 
focal defector, on the other hand, is 



Pn = 



ko + l 

kDi{k~l)-qc\nD+rk]+rk+kc 



(14) 

Consequently, pc increases by 1/A^ where N ^ L^, with 
probability 

Prob(Apc = i) = 

PD- E k^.d'c'iD^'iriD-nPc-PD), ^^^^ 

kc = l 
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where T{Pc — Pd) is the individual transition probability 
given by Eq. |9l The number of CC pairs increases by kc, 
and thus pcc increases by 2kc / (kN) with probability 



Prob(Apcc = f^) = 



(16) 



A cooperator, on the other hand, is selected for strategy up- 
dating with the probability pc- The frequency of a configu- 
ration that there are kc cooperators and kj:, defectors in the 
neighborhood of the focal cooperator is 



fe! kc ko 



(17) 



and the payoff of the focal cooperator is Pc{kc', ko) = kc- 
The configuration probability with which a neighboring coop- 
erator has k(j cooperators and kjj defectors as its neighbors 
is 



(fc-1)! 4 4 



(18) 



and the payoff of the neighboring cooperator is 
Pc{kc,kjj) ~ k(j + 1. Similarly, the configuration 
probability with which a neighboring defector has fc^ 
cooperators and kjj defectors as its neighbors is 



(fc-1)! ka 4 
k'^lk'^l '^c\dc1d\dc^ 



(19) 



and the payoff of the neighboring defector is Poikc, kjj) = 
(1 + r) ■ {kc + 1) + rkjj. Thus the average payoff Pc of 



cooperators in the neighborhood of the focal cooperator is 



'^c- E jh71''c^cc'll°cc-P'^('''c^'''D)+Pc{kc,kn) 



Pc 



kc-[{k~l)-qcicc+2] 
kc+1 ■ 



fcc+1 



(20) 

while, the average payoff Pd of defectors in the neighborhood 
of the focal cooperator is 



fc-i 



Pd - ll,k'^,'lc\Dc'lD\DC-PDikc^ ^d) ^21) 
= (fc - 1) • qc\DC + l + rk. 

Thus Pc decreases by 1 /N with probability 



Prob(Apc = -^) = 

PC ■ E k^.1c''\clD\C ■ - Pc). 

kc=0 



(22) 



Moreover, the number of CC pairs decreases by kc and thus 
Pcc decreases by 2kc/{kN) with probability 



Prob(Ap, 



CC 



PC-k^/c'ic'lDlC-TiPD-Pc). 



2kc ■ 
' kN . 



These derivations lead us to the master equations 



(23) 



PC = Prob(Apc' = ^) - Prob(Apc' = and 



N' 



N' 



(24) 



Pcc = X! -T^[Prob(Apcc 7^) - Prob(Apcc 



I — 



' kN 



(25) 



Although these equations are per derivation exact, they do 
depend on the density of triplet configurations which are 
outside their scope. Thus, in order to "close" the system of 
differential equations, the triplet configuration probabilities 
have to be approximated by probabilities of configurations 
that are not more complex than pairs. Note that by using 
different closure conditions, we can in general obtain different 
pair approximations. Here we employ the so-called ordinary 
pair approximation method, where only the first-order pair 
correlations are considered. We thus have qx\YZ ~ Qx\y- 
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